Characterization platform for scalable, spatially-resolved multispectral analysis of tissue

ABSTRACT

A device may obtain field images of a tissue sample, apply, to the field images, spatial distortion and illumination-based corrections (including corrections for photobleaching of reagents) to derive processed field images, identify, in each processed field image, a primary area including data useful for cell or subcellular component characterization, identify, in the processed field images, areas that overlap with one another, and derive information regarding a spatial mapping of cell(s) and/or sub-cellular components of the tissue sample. Deriving the information may include performing segmentation based on the data included in the primary area of each processed field image, and obtaining flux measurements based on other data included in the overlapping areas. The device may cause the information to be loaded in a data structure to enable statistical analysis of the spatial mapping for identifying factors defining normal tissue structure, associated inflammatory or neoplastic diseases and prognoses thereof, and associated therapeutics.

RELATED APPLICATIONS

This application is a 371 national stage of PCT Application No.PCT/US2019/051952 filed on Sep. 19, 2019, entitled “CHARACTERIZATIONPLATFORM FOR SCALABLE, SPATIALLY-RESOLVED MULTISPECTRAL ANALYSIS OFTISSUE,” which claims priority to U.S. Provisional Patent ApplicationNo. 62/734,737, filed on Sep. 21, 2018, which are hereby expresslyincorporated by reference herein.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under CA142779 awardedby the National Institutes of Health. The government has certain rightsin the invention.

BACKGROUND

Based on recent successes with immunotherapy, such as programmed celldeath-1 (PD-1)/programmed cell death ligand-1 (PD-L1) checkpointblockade, the number of clinical trials with new immunotherapies andimmunotherapy combinations is continuing to increase. Predictivebiomarkers, that can accurately stratify the likelihood of patientbenefit from a given immunotherapy and guide rational combinatorialstrategies, are in high demand. The analysis of the tumormicroenvironment in this context has illuminated the importance ofquantitative spatial profiling of normal and abnormal cells, andexpressed and secreted factors in healthy and abnormal tissues, indefining inflammatory and neoplastic disease and disease prognosis, aswell as therapeutic decision-making and resultant outcomes.

SUMMARY

According to some possible implementations, a method may includeobtaining, by a device, a plurality of field images of a specimen. Theplurality of field images may be captured by a microscope. The methodmay include processing, by the device, the plurality of field images toderive a plurality of processed field images. The processing may includeapplying, to the plurality of field images, spatial distortioncorrections and illumination-based corrections to address deficienciesin one or more field images of the plurality of field images. The methodmay include identifying, by the device and in each processed field imageof the plurality of processed field images, a primary area that includesdata useful for cell characterization or characterization of subcellularfeatures, identifying, by the device, areas of overlap in the pluralityof processed field images, and deriving, by the device, informationregarding a spatial mapping of one or more cells of the specimen.Deriving the information may be based on performing, by the device,image segmentation based on the data included in the primary area ofeach processed field image of the plurality of processed field images,and obtaining, by the device, flux measurements based on other dataincluded in the areas of overlap. The method may include causing, by thedevice and based on the information, an action to be performed relatingto identifying features related to normal tissue, diagnosis or prognosisof disease, or factors used to select therapy.

According to some possible implementations, a device may include one ormore memories, and one or more processors, communicatively coupled tothe one or more memories, configured to obtain a plurality of fieldimages of a tissue sample. The plurality of field images may be capturedby a microscope. The one or more processors may be configured to apply,to the plurality of field images, spatial distortion corrections andillumination-based corrections to derive a plurality of processed fieldimages, identify, in each processed field image of the plurality ofprocessed field images, a primary area that includes data useful forcell characterization, identify, in the plurality of processed fieldimages, areas that overlap with one another, and derive informationregarding a spatial mapping of one or more cells of the tissue sample.The one or more processors, when deriving the information, may beconfigured to perform segmentation, on a subcellular level, a cellularlevel, or a tissue level, based on the data included in the primary areaof each processed field image of the plurality of processed fieldimages, and obtain flux measurements based on other data included in theareas that overlap with one another, and cause the information to beloaded in a data structure to enable statistical analysis of the spatialmapping for identifying predictive factors for immunotherapy.

According to some possible implementations, a non-transitorycomputer-readable medium may store instructions. The instructions mayinclude one or more instructions that, when executed by one or moreprocessors, cause the one or more processors to obtain a plurality offield images of a tissue sample, apply, to the plurality of fieldimages, spatial distortion corrections and/or illumination-basedcorrections to derive a plurality of processed field images, identify,in each processed field image of the plurality of processed fieldimages, a primary area that includes data useful for cellcharacterization, identify, in the plurality of processed field images,areas that overlap with one another, and derive spatial resolutioninformation concerning one or more cells or subcellular components ofthe tissue sample. The one or more instructions, that cause the one ormore processors to derive the spatial resolution information, cause theone or more processors to perform image segmentation based on the dataincluded in the primary area of each processed field image of theplurality of processed field images, and obtain flux measurements basedon other data included in the areas that overlap with one another. Theone or more instructions, when executed by the one or more processors,may cause the one or more processors to cause a data structure to bepopulated with the spatial resolution information to enable statisticalanalyses useful for identifying predictive factors, prognostic factors,or diagnostic factors for one or more diseases or associated therapies.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1G are diagrams of an example implementation described herein.

FIG. 2 is a diagram of an example environment in which systems and/ormethods, described herein, may be implemented.

FIG. 3 is a diagram of example components of one or more devices of FIG.2 .

FIG. 4 is a flow chart of an example process for characterizing cells orsubcellular components of a specimen for statistical analysis.

FIG. 5 is a flow chart of an example process for characterizing cells orsubcellular components of a specimen for statistical analysis.

FIG. 6 is a flow chart of an example process for characterizing cells orsubcellular components of a specimen for statistical analysis.

DETAILED DESCRIPTION

The following detailed description of example implementations refers tothe accompanying drawings. The same reference numbers in differentdrawings may identify the same or similar elements.

Prognostic assays, that predict response/resistance to a givenimmunotherapy, involve spatial resolution of complex immunophenotypes.Multiplex biomarker development may reliably pinpoint the geography ofan expanded number of immunoactive factors in a tumor, or tumor immune,microenvironment. In-depth characterization of the microenvironment istypically achieved using formalin-fixed paraffin embedded (FFPE) tumorsamples or specimens (e.g., similar to those routinely received insurgical pathology). Existing techniques extend single antibodyimmuno-labeling approaches, of routine FFPE tissue sections, tomultilayered/multiplexed staining assays, using predominantlyimmunohistochemistry (IHC) and immunofluorescence (IF). However, thesetechniques typically analyze only a small number of high power fields orfield images, which limits the dataset used to characterize a tumorimmune microenvironment. Such techniques also involve a significantamount of manual human curation, which is time intensive, inefficient,error-prone, and unscalable, considering that large amounts of data(e.g., concerning tens of thousands of samples, tens of billions ofcells, and/or the like, with aggregate data volumes in petabytes) mayneed to be processed and analyzed. In fact, such techniques rely on theuse of spreadsheets for tracking information gleaned from field images,and while multi-antibody labeled tissue section scanning, using amicroscope, may take a small amount of time (e.g., about fifteenminutes), it may take a trained pathologist many hours to createspreadsheet-based training sets that identify the spatial organizationof key immunoactive/suppressive elements for use with subsequent cellsegmentation and classification. Further, thresholds of positivity(e.g., for determining whether signal intensity, corresponding to apotential marker, is sufficient to identify a positive marker) may vary,from specimen to specimen, as a function of pre-analytic variables, andexisting techniques rely on manual tuning of such thresholds (e.g., by apathologist), which is also inefficient and error-prone.

Some implementations, described herein, provide a characterizationplatform (e.g., an end-to-end system) that is capable of providing fast,reliable, and highly-scalable characterizations of spatially-resolvedinteractions at or below the single-cell level. In some implementations,the characterization platform may be configured to execute an automatedprotocol or pipeline for analyzing multiplex immunofluorescent and/orimmunohistochemistry data generated on tissue sections, including levelsof expression of multiple markers. In some implementations, theautomated pipeline may be implemented in various stages, includingstages for obtaining high-quality field images of a specimen (e.g., bycapturing, using a multispectral microscope, full sets of low-levelmulti-wavelength overlapping deep tissue field images), processing thefield images (e.g., to register the field images, correct spatialdistortions and illumination issues, and/or the like), deriving a mosaicusing the field images, performing image segmentation and obtaining fluxmeasurements (e.g., of cell markers, where appropriate colortransformations may be associated with different tissue types, whereavailable color information (e.g., all available color information) maybe used to aid tissue/cell segmentation and classification, and wheremachine learning techniques may be utilized to cluster the color spaceinto multiple regions that each corresponds to biologically meaningfulmorphological components for different tissue types), and developing aninteractive data structure for storing raw and processed field imagedata, outputs of the image segmentation, and the flux measurements. Insome implementations, the characterization platform may provide a userinterface that enables a user (e.g., a scientific researcher and/or thelike) to perform data analytics (e.g., using one or more functions, suchas spatial operation functions, integrated with the data structure) on,and visualize, the stored data. Further, in some implementations, thecharacterization platform may be configured to automatically determineoptimal thresholds of positivity for each individual specimen.

Providing an automated pipeline that is flexible and scalable, asdescribed herein, permits the collection and processing of a largeramount of data (e.g., a greater quantity of field images, obtained ineach of multiple optical bands) than possible with prior techniques,which increases analytical system throughput for aiding clinical trialsand clinical sample collection. In addition, automatically determiningoptimal thresholds of positivity for each individual specimen, asdescribed herein, also increases such throughput. Utilizing machinelearning techniques also streamlines the identification of cellcomponents, such as cellular nuclei, membranes, cytoplasms, and/or thelike. Automating the loading of a data structure (e.g., a parallel datastructure) with image segmentation outputs and flux measurements, andproviding spatial operation functions for statistically analyzing theloaded data, reduces or eliminates a need to rely on unwieldyspreadsheets, increases the accuracy and reproducibility of afully-automated cell classification system, and facilitatescharacterizations of interactions at the single-cell or sub-cellularlevel (e.g., including spatially-resolved measures of protein orMessenger Ribonucleic Acid (mRNA) expression). This provides faster andimproved insight into normal immune tolerance, how cancer evades theimmune system during development, immune-mediated side effects of newclasses of cancer therapies, and/or candidate therapeutic targets.

FIGS. 1A-1G are diagrams of an example implementation 100 describedherein. Example implementation 100 may include a microscope (e.g., amultispectral microscope with an image sensor or scanner) and acharacterization platform. As shown in FIG. 1A, the characterizationplatform may include, or otherwise have access to, a data structure. Forexample, the data structure may be implemented as a database, a table, atrie, a linked list, an array, and/or the like.

In some implementations, a specimen (e.g., a slice of a tumor tissueand/or the like) may be prepared in a microscope slide, and positionedunder the microscope for image capturing (e.g., for capturing of highpower fields or field images) by the microscope. In someimplementations, the microscope may be configured to capture multipleoverlapping, deep field images, covering a portion, or an entirety, ofthe specimen, in a redundant manner. Overlapping areas of the fieldimages may include duplicate data, such as multiple, repeat images ofthe same cells of the specimen. Such overlapping areas may be processed,for example, using various image processing operations (e.g., describedin more detail below) to improve any imaging-related deficiencies thatmay be inherent in the microscope. This redundant data can also be usedto characterize the accuracy of image segmentation and fluxmeasurements.

In some implementations, the characterization platform may include oneor more applications configured to execute a protocol or pipeline forimage processing, image segmentation, obtaining flux measurements, anddeveloping and/or populating a data structure, as described herein. Insome implementations, the characterization platform may be configured toexecute one or more stages of the pipeline automatically (e.g., thepipeline may be fully automated), based on user input, and/or the like.

As shown in FIG. 1A, and as shown by reference number 105, themicroscope (e.g., based on programmed instructions provided by thecharacterization platform, based on user input, and/or the like) maycapture multiple sets of field images of the specimen. For example, themicroscope may capture a large quantity of field images, such as about1,500 field images, about 2,000 field images, and/or the like. In someimplementations, the microscope may capture field images across anentirety of the specimen. In some implementations, the microscope maycapture field images that overlap (e.g., where a given object (e.g., acell) may be imaged multiple times over multiple field images such thatthere are overlapping areas in the field images). In someimplementations, the microscope may capture field images using multipleoptical bands (e.g., light at different wavelengths or in differentwavelength ranges). For example, light filters, corresponding tomultiple wavelength ranges (e.g., thirty-five narrow wavelength rangesand/or the like) may be sequentially used to capture multiple sets offield images—e.g., capturing about 2,000 field images at each ofthirty-five wavelength ranges may result in a total of about2,000×35=70,000 field images.

As shown by reference number 110, the characterization platform mayreceive the sets of field images from the microscope. In someimplementations, the characterization platform may be communicativelycoupled to the microscope (e.g., to a processor and/or memory of themicroscope) to control image capturing and/or to receive captured fieldimages. As shown by reference number 115, the characterization platformmay subject the sets of field images to various image processingoperations. In some implementations, the image processing operations mayinclude corrections of distortions in the field images of each set offield images. For example, overlapping areas in the field images maycorrespond to different sections of a field of view of the microscope'simage sensor, and thus exhibit spatial distortions (e.g., warping due toartifacts of the microscope) that are different across the various fieldimages. In some implementations, the characterization platform may beconfigured to iteratively develop a uniform correction model of spatialdistortions for the field images in each set of field images, and applycorrections that align the field images (e.g., accurate to a fractionalsize of a pixel and/or the like) with one another to unwarp each fieldimage. In some implementations, the characterization platform maycross-correlate overlapping areas of the field images to determineglobal placement of the field images, where, for example, warping and/orrelative shifts between field images is minimal. In someimplementations, cross-correlation may include determining mismatchesbetween field images (e.g., relative shifts between field images),associating such mismatches with a virtual spring variable to identifyhow much a virtual spring is extended, and determining an appropriateposition for each of the field images based on extensions of the virtualsprings (e.g., based on identifying a center field image in a set offield images, and permitting cross-correlation to settle the fieldimages, in the set of field images, into equilibrium).

For example, in some implementations, the characterization platform maybe configured to determine optimal relative translations (e.g., shifts)in (x, y) positions of each pair of overlapping field images. In anarray of field images, there may be 4-connected overlaps among the fieldimages, 8-connected overlaps among the field images (e.g., in corners ofthe field images, which may contain fewer pixels, and may have a higherpossibility of warping errors, than 4-connected overlaps), and/or thelike. In some implementations, the characterization platform maydetermine updated centers of the field images, to minimize the relativeshifts, using a model of elastic springs. For example, assume that eachoverlap in the field images corresponds to a spring that is stretched byan amount corresponding to the relative shifts between the field images,and that an array of field images is connected with a set of springscurrently stretched by a different amount (or length). By pinning one ofthe field images to a center of the microscope slide, the remainingfield images may be pulled into equilibrium as the springs settle. Thisequilibrium configuration may correspond to a minimum total springenergy, given the virtual spring connections and initial stretchesthereof (e.g., as shown in FIG. 1F). Here, the energy of a singleelastic spring may be defined as:E(x)=1/2Dx ²   (1)where D represents the spring constant and x, in this equation,represents the length of the spring. The entire system may beillustrated via a graph, where nodes (N) of the graph are the fieldimages, and edges (E) are the overlaps of the field images. In somecases, only the 4-connected overlaps (and not the 8-connected overlaps)may be considered. Here, the initial energy of the entire system, in oneof the dimensions (e.g., in the x dimension) may be defined as:

$\begin{matrix}{E = {\frac{1}{2}D{\sum\limits_{{({u,\nu})} \in E}\left( s_{u\nu} \right)^{2}}}} & (2)\end{matrix}$where s_(uv) represents the empirically measured displacement betweenfield images, or nodes, u and v, along the x direction. The summationmay be over all edges between nodes u and v, where (u, v) forms anoverlapping pair. After adjusting the nominal field center of node u byan amount (e.g., a small amount), x(u), the spring energy may bemodified to:

$\begin{matrix}{E = {\frac{1}{2}D{\sum\limits_{{({u,v})} \in E}\left( {s_{u\nu} - {x(u)} + {x(v)}} \right)^{2}}}} & (3)\end{matrix}$

To select an amount, x(u), that minimizes the total energy of the entiresystem, a partial derivative, with respect to each x(u), may becalculated and equated to equation (1) above. The value of D may also beassumed to be ‘1’ to simplify the notation.

$\begin{matrix}{\frac{\partial E}{\partial{x(w)}} = {\sum\limits_{{({u,v})} \in E}{\left( {s_{uv} - {x(u)} + {x(v)}} \right)\frac{\partial}{\partial{x(w)}}\left( {s_{uv} - {x(u)} + {x(v)}} \right)}}} & (4) \\{\frac{\partial E}{\partial{x(w)}} = {\sum\limits_{{({u,v})} \in E}{\left( {s_{uv} - {x(u)} + {x(v)}} \right)\left\lbrack {{- \delta_{uw}} + \delta_{vw}} \right\rbrack}}} & (5)\end{matrix}$

Subsequently, a sum of the Kronecker deltas may be calculated. As shownin the summation notation below, u: (uw)∈E may indicate that a sum istaken over all nodes u that overlap node w.

$\begin{matrix}{\frac{\partial E}{\partial{x(w)}} = {{\sum\limits_{u:{{({u,w})} \in E}}\left( {s_{uw} - {x(u)} + {x(w)}} \right)} - {\sum\limits_{v:{{({w,v})} \in E}}\left( {s_{wv} - {x(w)} + {x(v)}} \right)}}} & (6)\end{matrix}$In a simpler form, this may be represented as follows (e.g., if aso-called adjacency matrix A is defined):

$\begin{matrix}{A_{uv} = \left\{ \begin{matrix}{1,} & {\left( {u,v} \right) \in E} \\{0,} & {else}\end{matrix} \right.} & (7)\end{matrix}$Further, a degree of a node, d(u), may be defined as a quantity of edges(e.g., overlaps) of node u. Here, the displacements s_(uv) may beantisymmetric:s _(uv) =−s _(vu)   (8)The u and v sums over connected nodes may be the same:

$\begin{matrix}{\frac{\partial E}{\partial{x(w)}} = {2\left( {{{d(w)}{x(w)}} - {\sum\limits_{v}{A_{vw}{x(v)}}} + {\sum\limits_{u:{{({u,w})} \in E}}s_{uw}}} \right)}} & (9)\end{matrix}$The Laplacian matrix L=D−A may then be introduced, where D, here,represents the diagonal matrix formed out of the degrees of each node. Avector S(w), which represents the sum along each row of s_(wu), may alsobe defined:

$\begin{matrix}{{S(w)} = {\sum\limits_{u:{{({wu})} \in E}}s_{wu}}} & (10)\end{matrix}$This may yield a simpler equation:

$\begin{matrix}{{\frac{1}{2}\frac{\partial E}{\partial{x(w)}}} = {{{\sum\limits_{u}{L_{wu}{x(u)}}} - {S(w)}} = 0}} & (11)\end{matrix}$

The system of equations may have an infinite quantity of solutions,since all fields are floating, and thus an entire stable configurationmay be translated by an arbitrary amount, without altering the energy.This may be resolved by pinning one of the fields, e.g., with a label z.Furthermore, the displacement of this field may be specified as x(z)=0.In this case, the z-th row and column of L, and the z-th element of Sand x, may be removed to yield the matrix {circumflex over (L)} andvectors Ŝ and {circumflex over (x)}. The set of linear equations maythen include:{circumflex over (L)}{circumflex over (x)}=Ŝ  (12)with a solution:{circumflex over (x)}={circumflex over (L)} ⁻¹ Ŝ  (13)The foregoing optimization may be performed in both dimensions (e.g., inthe x dimension and the y dimension) as part of determining the optimalshifts for the field images.

In some cases, illumination, during image capturing by the image sensorof the microscope, may not be uniform across a field image. Thus, insome implementations, the image processing operations may includeapplying an illumination correction to overlapping areas of the fieldimages—e.g., to calculate properly flat-fielded images where the fieldimages have uniform, or substantially uniform, pixel intensities. Insome implementations, photobleaching of reagents/tissue may occur due toacquisition of multiple overlapping field images. In such cases, theimage processing operations may include corrections for suchphotobleaching. Alternatively, in some implementations, photobleachingeffects may not be corrected, but may rather be leveraged tocharacterize and/or define cell types, subcellular component types,and/or the like.

In some implementations, the image processing operations may include abackground subtraction process and/or the like. In some implementations,the characterization platform may, after subjecting the sets of fieldimages to the above-described image processing operations, arrive at anoverlapping mesh of field images.

In some implementations, the characterization platform may export, orsave, each processed field image (e.g., registered, corrected forwarping, flat-fielded, and/or the like) to a standard format (e.g., as atagged image file format (TIFF) file, a portable network graphics (PNG)file, a bitmap (BMP) file, and/or the like) to enable subsequent imagesegmentation and obtaining of flux measurements (e.g., described in moredetail below).

In some implementations, the characterization platform may identify aprimary area, in each processed field image, that includes pixels of thehighest quality (e.g., pixels that have been subjected to few, or none,of the above-described spatial distortion and/or illuminationcorrections). In some implementations, a set of pixels, positionedclosest to a center point in a processed field image, may be consideredas being part of a primary area of the processed field image. In someimplementations, the characterization platform may define a boundaryrectangle that includes and/or encloses the set of pixels to distinguishthe primary area from a remainder of the processed field image (e.g., aremainder that may include an overlapping area of the processed fieldimage). In various implementations, the primary areas of the processedfield images may include the most pertinent data useful for statisticalanalysis and study. In some implementations, non-primary areas (e.g.,the overlapping areas) of the processed field images may be utilizedonly for quality control (e.g., analyzed to compensate for errors inmeasurements provided by the microscope).

Returning to FIG. 1A, and as shown by reference number 120, thecharacterization platform may derive a mosaic based on the processedfield images. For example, the characterization platform may derive themosaic based on the primary areas identified in all the processed fieldimages (e.g., by positioning the primary areas, edge-to-edge). Anexample mosaic is shown in FIG. 1D (color information not portrayed). Asshown, the mosaic may be a large image (e.g., including about 60,000pixels by about 40,000 pixels and/or the like) composed of seamlesstiles or fields. A field may refer to a subset of or a portion of alarger image. For example, a field and/or boundaries of a field may bedefined by a box, a polygon, a circle, an ellipse, a rectangle, a strip,and/or another shape. Providing a large mosaic, that includes theprimary areas (e.g., the highest quality pixels) of all the processedfield images, and thus the most statistically uniform, data for aportion, or an entirety, of a specimen, enables further analysis andderivation of additional data as needed (e.g., such as that which may beuseful for display as visual overlays, as described in more detailbelow).

As shown in FIG. 1B, and as shown by reference number 125, thecharacterization platform may perform image segmentation and obtain fluxmeasurements (e.g., of markers, such as cell markers) based on theprocessed field images. In some implementations, the characterizationplatform may perform image segmentation to identify pixels, in theprimary areas of the processed field images, that correspond to cells(e.g., some or all of the cells) and/or subcellular components (e.g.,some or all of the subcellular components) of the specimen. In someimplementations, the characterization platform may, as part of imagesegmentation, utilize one or more image analysis and/or deep learningtechniques (e.g., deep convolutional neural networks) to analyze themultispectral data in the processed field images. In someimplementations, a 10-12 dimensional principal component analysis (PCA)color space may be defined and utilized in the image segmentation.

In some implementations, the characterization platform may identify thepixels that correspond to each cell and/or subcellular component,identify a center of each cell, determine an outline of each cell and/orsubcellular component, classify each cell and/or subcellular componentby type (e.g., as a tumor cell, as an immune cell, and/or the like thatmay be relevant for one or more immunotherapy reactions), determine anoutline of a tumor, determine various spatially objective quantities(e.g., distances between cells, distances between subcellularcomponents, and/or the like), and/or the like. In some implementations,the characterization platform may associate a classification probabilitywith each cell and/or subcellular component, which enablesidentification, filtering, and/or improved cell and/or subcellularclassification via Bayesian techniques and/or the like.

In some implementations, the characterization platform may combinecellular positions with protein or mRNA expression levels and/or thelike of the markers, and map such cellular positions to sub-cellularcompartments (e.g., nuclear, cytoplasmic, and membranous expression(s)).In some implementations, the characterization platform may determinecell phenotypic data for an immune cell subset, determine spatialposition information of a given cell relative to a tumor and otherimmune cell subsets, and compute spatial relations between immune cellsand tumor cells (e.g., conditioned on expression levels of severaldifferent markers, and on positions of such markers relative to tissueboundaries). This permits, for example, correlation of tissuearchitectural features with clinical parameters, such as age, survival,therapies received, and/or the like, as well as with other profilingfeatures, such as genomics, transcriptomics, metabolomics, patientmicrobiome, and/or the like.

In some implementations, the characterization platform may obtain fluxmeasurements, for various markers, based on overlapping areas of theprocessed field images. Overlapping field images may provide duplicateinformation (e.g., at least two times the information, and in somecases, four times the information) for a given cell or subcellularcomponent—e.g., multiple cell-related measurements that may bedifferent—which yields error bar information that may be useful forquantifying the accuracy of the image capturing process and theabove-described image segmentation. In a sample case, the variance ofmeasured flux differences, between overlapping areas of processed fieldimages, is improved over variances obtained using existing techniques(e.g., improved by a factor of about 3.0 for PD-1 and by a factor ofabout 4.0 for PD-L1). In some implementations, uncertainty estimates maybe quantified over narrowly defined samples (e.g., conditioned on celltype, subcellular component type, and/or the like).

In this way, the characterization platform may analyze the processedfield images and perform associated measurements, in a highly-scalableand automated manner, to arrive at a spatial mapping of cellularsubsets, secreted or transcription factors, mRNA expression,deoxyribonucleic acid (DNA) alterations (e.g., including, but notlimited to, chromosomal alterations (e.g., translocations,amplifications, and/or deletions) observable via fluorescent in situhybridization and/or similar techniques) in normal tissues and/orassociated inflammatory conditions or tumors. This may involveprocessing tens of thousands of field images, and characterizingbillions of cells and/or subcellular components, in a manner that cannotbe performed manually or objectively by a human actor.

As shown by reference number 130, the characterization platform maystore, or load, information, regarding outputs of the image segmentationand/or the flux measurements, in the data structure. In someimplementations, the data structure may be configured to store a largeamount of data (e.g., petabytes of data and/or the like).

In some implementations, the characterization platform may storeinformation regarding a position of each cell and/or subcellularcomponent, information regarding spatial coordinates of each cell and/oreach subcellular component, information regarding an outline of eachcell and/or each subcellular component (e.g., morphological datarepresented as a polygon and/or the like), information regarding acenter of each cell, information regarding distances between cells(e.g., an immune cell and one or more nearby tumor cells), informationregarding distances between subcellular components, informationregarding distances between cells and a boundary of a tissue sample,information regarding distances between subcellular components and theboundary of the tissue sample, information regarding variable,dynamically-adjusted boundaries and/or contours within a specimen thatmay or may not represent any anatomic boundaries), information regardingflux measurements of various cell markers, information regardingmulti-channel intensity measurements, and/or the like. This enablesspatial operations to be performed on any of such information for avariety of statistical analyses (e.g., to determine if T-cells havepenetrated into a tumor and/or the like).

In some implementations, the characterization platform may define aconsistent global coordinate system (e.g., in microns) across themicroscope slide, and transform a two-dimensional (2D) spatial positionof each cell to the global coordinate system. Such a global coordinatesystem may also allow for 3D arrangements of multiple slide images andalignment of such images in a z-plane.

In some implementations, loading of the information into the datastructure may be performed in two phases (e.g., including hierarchicalerror handling and reporting). For example, in some implementations, thecharacterization platform may perform validation and/or cleaning of theinformation prior to storing the information in the data structure.Continuing with the example, the characterization platform may initiallystore the information in a temporary data structure, process theinformation (e.g., by scrubbing the information to identify errors,verify consistency, and/or the like), and load the processed informationinto the data structure. This may avoid contamination of any existingdata in the data structure with erroneous, or non-optimal, data.

In some implementations, the characterization platform may store, in thedata structure, the originally-captured field images, the processedfield images (e.g., registered, corrected, and/or the like, as describedabove), an image pyramid of the originally-captured field images andprocessed field images, information regarding an experimental fieldlayout, and/or the like, which may allow for visual integration (ofmarkers, e.g., at multiple resolutions) with other data in the datastructure. In some implementations, the characterization platform maystore information regarding a historical origin (e.g., a provenance) ofeach cell to a file or file structure corresponding to the cell. In someimplementations, the characterization platform may store theabove-described mosaic in the data structure. Additionally, oralternatively, the characterization platform may analyze the mosaic todetermine spatial feature data (e.g., locations of cells and/orsubcellular components, properties of cells and/or subcellularcomponents, and/or the like), and store such spatial feature data in thedata structure. In some implementations, the characterization platformmay overlay some or all of such spatial feature data over the fieldimages when the field images are displayed.

In some implementations, the characterization platform may storeuser-generated markups and/or annotations (e.g., generated during, orafter, image segmentation and/or obtaining flux measurements, forexample, to exclude fields of inferior image quality or highlight fieldsof specific interest). In some implementations, the characterizationplatform may store metadata that supports searching and/or spatialoperations on the data in the data structure. For example, the metadatamay describe contents of the data in the data structure, units ofmeasurements that are used, and/or the like. In some implementations,the data structure may include (e.g., at multiple levels of granularity)internal tables containing a text-based description of each column, oneor more objects (e.g., database objects), and/or the like. In someimplementations, the characterization platform may automaticallygenerate certain metadata by parsing a schema of the data structure(e.g., a database schema).

In some implementations, the characterization platform may store a flag,for each measurement relating to a cell and/or subcellular component,indicating whether the cell and/or subcellular component is included ina primary area of a processed field image or in an overlapping areathereof. This permits the characterization platform to distinguishbetween data that may be analyzed or spatially operated upon (e.g., dataassociated with primary areas of processed field images, which have thehighest quality pixel information useful for statistical analyses) anddata that may be used mainly for error estimations and quality control(e.g., data associated with overlapping areas of the processed fieldimages).

In some implementations, the data structure may include a set of indexesfor optimal search functionality and performance. For example, theindexes may represent spatial positions, spatial relations, and/or thelike.

In various implementations, and as briefly described above, the datastructure may support functions that enable a user (e.g., a scientificresearcher and/or the like) to perform various analytics on the datastored in the data structure. In some implementations, the functions mayinclude user-defined functions relating to analytics patterns (e.g.,low-level or mid-level analytics). For example, the functions may relateto spatial searches, operations on spatial polygons (e.g., operationsassociated with identifying unions, identifying intersections,identifying differences, shrinking polygons, growing polygons, and/orthe like), custom aggregations subject to various conditions (e.g., cellphenotypes, subcellular phenotypes, expression levels, spatialrelations, and/or the like) optimized for analytics and research, and/orthe like. In some implementations, the functions may be capable ofrepresenting (e.g., operating on or causing to be visually presented)spatial polygons relating to tissue boundaries, morphological featuresof cells, shapes of morphological components of cells (e.g., thenucleus, the membrane, and/or the cytoplasm). In some implementations,the functions may enable calculations of spatial distances betweencells, spatial distances between subcellular components, spatialdistances between cells and tissue boundaries, spatial distances betweensubcellular components and tissue boundaries, and/or the like. In someimplementations, the functions may be capable of representing (e.g.,operating on or causing to be visually presented) markers associatedwith cell components and/or the like.

As shown in FIG. 1C, and as shown by reference number 135, thecharacterization platform may provide a user interface configured topermit a user to navigate, and execute searches and/or analytics on, thedata stored in the data structure. In some implementations, the userinterface may be implemented as a scalable interactive browser thatpermits manipulation (e.g., zooming in (e.g., down to the highestresolution) and out, panning, and/or the like) of field images and/orwhole-tissue images (e.g., a mosaic) stored in the data structure. Insome implementations, the user interface may be configured to presentvarious types of data stored in the data structure (e.g., relating tomorphological components of cells, spatial features, and/or the like,which may be learned from the image segmentation, the flux measurements,and/or the mosaic) as an overlay.

In some implementations, the data structure may support queries (e.g.,low-level database queries), and may be integrated with statisticalpackages (e.g., high-level packages, such as R, MATLAB, Python, and/orthe like) that include distribution functions and/or functionsconfigured to characterize intercellular distances relating to varioussubsets. This enables data analytics (e.g., directly in the datastructure environment) without needing to copy data from the datastructure to an external computing device for analysis.

In some implementations, the characterization platform may be configuredto operate on the data in the data structure to mark individual cells,subcellular components, or regions of interest that satisfy certaincriteria. In some implementations, the user interface may be configuredto present such marked regions, which may facilitate spatial harvestingfor single-cell transcriptomic or genomic studies. In someimplementations, the user interface may provide a user-selectable optionto export lists of marked regions for further review or study.

In addition to enabling the characterization of cells and/or subcellularcomponents, the characterization platform may also enablecharacterizations of tumor and immune interactions, such as relating toPD-1, PD-L1, and/or the like. In some implementations, various fluxmeasurements may be used to determine whether activation of markers forsuch interactions is positive or negative. In some implementations, thecharacterization platform may utilize individual image layers fordetecting each such marker (e.g., to identify cells that may have anoverlay in a color corresponding a marker, and determine whether anintensity of the overlay satisfies a threshold of positivity). Forexample, an intensity that does not satisfy the threshold of positivitymay be indicative of noise. In some implementations, thecharacterization platform may include an algorithm configured todetermine an optimal threshold of positivity for each marker of eachspecimen of interest. This permits the characterization platform toaccurately, and automatically, identify markers, which may facilitatestudies of cellular interactions and associated factors (e.g., studiesvia deep learning and/or the like). For example, in someimplementations, the characterization platform may be configured toutilize the identified information to automatically generate (e.g.,using one or more deep learning algorithms and/or the like)recommendations for one or more particular diagnoses or therapeuticsand/or the like.

In some implementations, the algorithm may determine differences betweenpositive and negative results (e.g., corresponding to marker signal andnoise, respectively), and identify an intersection of the results todetermine an optimal threshold of positivity. FIG. 1E is a diagram thatillustrates a mathematical model for determining an optimal threshold ofpositivity for a particular specimen of interest. As shown in FIG. 1E,curve A is a distribution of all the intensities of the cells of aspecimen. Assuming that noise is approximately Gaussian, the algorithmmay be configured to subtract such noise from curve A to arrive at curveB, and determine an optimal threshold of positivity based on anintersection of curve A and curve B. Applying such an optimal thresholdof positivity enables proper cell identification and quantification(e.g., counting). Additionally, or alternatively, an algorithm may beconfigured to determine an optimal threshold of positivity by setting afirst threshold of positivity for a first field image, setting a secondthreshold of positivity for a second field image, performing acomparison of overlapping area(s) of the two field images, utilizing aresult of the comparison to adjust the first threshold of positivityand/or the second threshold of positivity, and repeating the comparisonuntil a predefined result is achieved (e.g., where the same markeridentification results are obtained for both field images). In any case,in some implementations, the characterization platform may provide auser-selectable option to adjust a value of the threshold of positivity,as needed.

In some implementations, the characterization platform may analyzefields (or tiles) of a derived mosaic of a tissue image to identifyfields in which a feature (e.g., a marker, a biomarker, an anatomiccharacteristic such as a vessel, and/or the like) is present.Additionally, or alternatively, the characterization platform maydetermine a level of this feature in each field of a set of analyzedfields. The level of the feature may represent, for example, a densityof the marker within the field, a level of expression of the markerwithin the field, and/or the like. The characterization platform maysort and/or rank the fields according to the level of expression, andmay select a threshold number of fields (e.g., 20 fields, 25 fields, 30fields, either based on marker expression or a random sampling) or athreshold percentage of fields (e.g., of all analyzed fields, of allfields exhibiting the marker, of all fields in the mosaic) with adesired level of expression (e.g., the highest marker densities). Thecharacterization platform may further analyze the selected fields topredict responsiveness of the tissue to one or more types ofimmunotherapies. For example, the characterization platform may analyzethe selected fields for another feature, and the level of this otherfeature may indicate a predicted level of responsiveness toimmunotherapy.

In some implementations, the characterization platform may use a firstfeature to select the fields, and may use a second feature to analyzethe selected fields to predict responsiveness to immunotherapy. As anexample, and as shown in FIG. 1G, the first feature may be a CD8 marker,and the second feature may be combined FoxP3CD8 or CD163 PD-L1neg,FoxP3CD8 PDL1+mid, Other cells PD1low+, PDL1+, FoxP3CD8+PD1+mid, CD 163PDL1+, and/or the like. Although two features are provided as anexample, a larger number of features may be used in someimplementations. Additionally, or alternatively, other features may beused as the first feature and/or the second feature, such as an anatomiccharacteristic (e.g., a vessel), a location within the tumormicroenvironment (e.g., a peritumoral region), a staining characteristicof a marker, a cell expression of a marker, and/or the like.Furthermore, while a level of a feature is described above as thetechnique for selecting fields and analyzing those fields forimmunotherapy responsiveness, other techniques may be used, such as alevel of expression of a marker, co-expression of multiple markers,spatial proximity of the same marker, spatial proximity of differentmarkers, and/or the like. In FIG. 1G, about 25 fields are selected usingthe CD8 marker, and the FoxP3CD8 marker is used to predict immunotherapyresponsiveness with a threshold degree of confidence. As shown in FIG.1G, in this example, the maximum area under the curve is 0.74, asindicated by the oval in FIG. 1G, and is dependent on the number offields selected.

In some implementations, the characterization platform may analyzedistances between centroids of cells with particular markers, such as adistance between a centroid of a first cell marked with a first markerand a centroid of a second cell marked with a second marker. Asindicated above, the first marker and the second marker may include anycombination of CD163, FoxP3, CD163 PDL1neg, Tumor, Tumor PDL1+mid,FoxP3CD8PD1+low, FoxP3 PD1low+PDL1+, FoxP3CD8 PDL1+mid, Other cellsPD1low+, PDL1+, FoxP3CD8+PD1+mid, CD 163 PDL1+, and/or other markers. Insome implementations, the characterization platform may determine adistance or distances between centroids and may predict responsivenessto immunotherapy based on the distance or distances, such as bypredicting immunotherapy responsiveness based on the distances beingless than or equal to a threshold (e.g., 20 microns, 25 microns, 30microns, and/or the like). For example, if the characterization platformdetermines that the first feature or marker and the second feature ormarker are present or expressed and are within a threshold distance ofone another, then the characterization platform can predictimmunotherapy responsiveness with a threshold degree of confidence.Although two markers are provided as an example, a larger number ofmarkers may be used in some implementations.

Thus, the fields may be selected based on ranking fields for which afeature or a combination of features satisfies a first condition (e.g.,a condition relating to a level of expression or co-expression, acondition relating to spatial proximity or markers, a condition relatingto another tissue feature, and/or the like). The selected fields maythen be analyzed for responsiveness to immunotherapy based on whether amarker or a combination of markers satisfies a second condition.Although two conditions are provided as an example, a larger number ofconditions may be used in some implementations. In this way, thecharacterization platform may perform an automated analysis on a tissueimage to predict responsiveness to immunotherapy.

Providing an automated pipeline that is flexible and scalable, asdescribed herein, permits the collection and processing of a largeramount of data (e.g., a greater quantity of field images, obtained ineach of multiple optical bands) than possible with prior techniques,which increases analytical system throughput for possibleclinical-decision making. In addition, automatically determining optimalthresholds of positivity for each individual specimen as well asautomated field selection, as described herein, also increases suchthroughput. Utilizing machine learning techniques also streamlines theidentification of cell components, such as cellular nuclei, membranes,cytoplasms, and/or the like. Automating the loading of a data structure(e.g., a parallel data structure) with image segmentation outputs andflux measurements, and providing spatial operation functions forstatistically analyzing the loaded data, reduces or eliminates a need torely on unwieldy spreadsheets, increases the accuracy andreproducibility of a fully-automated cell classification system, andfacilitates characterizations of interactions at the single-cell level(e.g., including spatially-resolved measures of subcellular components,such as protein, mRNA, or cytokine expression). This provides faster andimproved insight into normal tissue function, inflammatory andneoplastic disorders, and potential diagnosis, prognosis, therapeuticinterventions.

As indicated above, FIGS. 1A-1G are provided merely as examples. Otherexamples are possible and may differ from what was described with regardto FIGS. 1A-1G. For example, although some implementations are describedherein with respect to immuno-oncology applications, the implementationsare equally, or similarly, applicable for use with prognostic featuresand therapeutic modalities across a broad array of malignant diseases.Additionally, or alternatively, although some implementations aredescribed herein as using a particular number of features or markers(e.g., two features or markers) to select fields for further analysisand/or to predict responsiveness to immunotherapy, a larger numberfeatures or markers may be used in some implementations.

FIG. 2 is a diagram of an example environment 200 in which systemsand/or methods, described herein, may be implemented. As shown in FIG. 2, environment 200 may include a characterization platform 210, a cloudcomputing environment 220, microscope device(s) 230, a user device 240,and a network 250. Devices of environment 200 may interconnect via wiredconnections, wireless connections, or a combination of wired andwireless connections.

Characterization platform 210 includes one or more devices capable ofreceiving, generating, storing, processing, and/or providing informationassociated with a specimen (e.g., slices, or samples, of tissue).Characterization platform 210 may include a server device or a group ofserver devices. In some implementations, as shown, characterizationplatform 210 can be hosted in cloud computing environment 220. Notably,while implementations described herein describe characterizationplatform 210 as being hosted in cloud computing environment 220, in someimplementations, characterization platform 210 is not cloud-based or canbe partially cloud-based.

Cloud computing environment 220 includes an environment that deliverscomputing as a service, whereby shared resources, services, etc. can beprovided to microscope device(s) 230, user device 240, and/or one ormore other characterization platforms 210. Cloud computing environment220 can provide computation, software, data access, storage, and/orother services that do not require end-user knowledge of a physicallocation and configuration of a system and/or a device that delivers theservices. As shown, cloud computing environment 220 can include a set ofcomputing resources 222.

Computing resource 222 includes one or more personal computers,workstation computers, server devices, or another type of computationand/or communication device. In some implementations, computing resource222 can host characterization platform 210. The cloud resources caninclude compute instances executing in computing resource 222, storagedevices provided in computing resource 222, data transfer devicesprovided by computing resource 222, etc. In some implementations,computing resource 222 can communicate with other computing resources222 via wired connections, wireless connections, or a combination ofwired and wireless connections.

As further shown in FIG. 2 , computing resource 222 can include a groupof cloud resources, such as one or more applications (“APPs”) 222-1, oneor more virtual machines (“VMs”) 222-2, virtualized storage (“VSs”)222-3, one or more hypervisors (“HYPs”) 222-4, and/or the like.

Application 222-1 includes one or more software applications that can beprovided to or accessed by microscope device(s) 230 and/or user device240. Application 222-1 can eliminate a need to install and execute thesoftware applications on microscope device(s) 230 and/or user device240. For example, application 222-1 can include software associated withcharacterization platform 210 and/or any other software capable of beingprovided via cloud computing environment 220. In some implementations,one application 222-1 can send/receive information to/from one or moreother applications 222-1, via virtual machine 222-2.

Virtual machine 222-2 includes a software implementation of a machine(e.g., a computer) that executes programs like a physical machine.Virtual machine 222-2 can be either a system virtual machine or aprocess virtual machine, depending upon use and degree of correspondenceto any real machine by virtual machine 222-2. A system virtual machinecan provide a complete system platform that supports execution of acomplete operating system (OS). A process virtual machine can execute asingle program, and can support a single process. In someimplementations, virtual machine 222-2 can execute on behalf ofmicroscope device(s) 230, a user (e.g., user device 240), and/or one ormore other characterization platforms 210, and can manage infrastructureof cloud computing environment 220, such as data management,synchronization, or long-duration data transfers.

Virtualized storage 222-3 includes one or more storage systems and/orone or more devices that use virtualization techniques within thestorage systems or devices of computing resource 222. In someimplementations, within the context of a storage system, types ofvirtualizations can include block virtualization and filevirtualization. Block virtualization can refer to abstraction (orseparation) of logical storage from physical storage so that the storagesystem can be accessed without regard to physical storage orheterogeneous structure. The separation can permit administrators of thestorage system flexibility in how the administrators manage storage forend users. File virtualization can eliminate dependencies between dataaccessed at a file level and a location where files are physicallystored. This can enable optimization of storage use, serverconsolidation, and/or performance of non-disruptive file migrations.

Hypervisor 222-4 provides hardware virtualization techniques that allowmultiple operating systems (e.g., “guest operating systems”) to executeconcurrently on a host computer, such as computing resource 222.Hypervisor 222-4 can present a virtual operating platform to the guestoperating systems, and can manage the execution of the guest operatingsystems. Multiple instances of a variety of operating systems can sharevirtualized hardware resources.

Microscope device(s) 230 include one or more devices capable ofcapturing images of specimens on a microscope slide. For example, amicroscope device 230 may include, or have access to, an image sensor(e.g., a color camera) configured to capture field images (e.g., highpower fields) of specimens. In some implementations, a microscope device230 may include multiple lenses, a processor device, and one or moremotors configured to control movement of the lenses for focusing ondifferent areas of a specimen and to control operation of the imagesensor for deep imaging of the different areas. In some implementations,a microscope device 230 may include a microspectral microscope thatenables capturing of field images using one or more of multiple opticalbands. In some implementations, field images, captured by a microscopedevice 230, may be provided to a characterization platform, such ascharacterization platform 210, for processing, as described elsewhereherein.

User device 240 includes one or more devices capable of receiving,generating, storing, processing, and/or providing information associatedcharacterization platform 210. For example, user device 240 may includea communication and/or computing device, such as a mobile phone (e.g., asmart phone, a radiotelephone, etc.), a desktop computer, a laptopcomputer, a tablet computer, a handheld computer, a gaming device, awearable communication device (e.g., a smart wristwatch, a pair of smarteyeglasses, etc.), or a similar type of device. In some implementations,user device 240 may receive statistical analysis data and/or resultsfrom characterization platform 210, and present such data and/or resultsfor display.

Network 250 includes one or more wired and/or wireless networks. Forexample, network 250 may include a cellular network (e.g., a long-termevolution (LTE) network, a code division multiple access (CDMA) network,a 3G network, a 4G network, a 5G network, another type of nextgeneration network, etc.), a public land mobile network (PLMN), a localarea network (LAN), a wide area network (WAN), a metropolitan areanetwork (MAN), a telephone network (e.g., the Public Switched TelephoneNetwork (PSTN)), a private network, an ad hoc network, an intranet, theInternet, a fiber optic-based network, a cloud computing network, and/orthe like, and/or a combination of these or other types of networks.

The number and arrangement of devices and networks shown in FIG. 2 areprovided as an example. In practice, there may be additional devicesand/or networks, fewer devices and/or networks, different devices and/ornetworks, or differently arranged devices and/or networks than thoseshown in FIG. 2 . Furthermore, two or more devices shown in FIG. 2 maybe implemented within a single device, or a single device shown in FIG.2 may be implemented as multiple, distributed devices. Additionally, oralternatively, a set of devices (e.g., one or more devices) ofenvironment 200 may perform one or more functions described as beingperformed by another set of devices of environment 200.

FIG. 3 is a diagram of example components of a device 300. Device 300may correspond to characterization platform 210, microscope devices 230,and/or user device 240. In some implementations, characterizationplatform 210, microscope devices 230, and/or user device 240 may includeone or more devices 300 and/or one or more components of device 300. Asshown in FIG. 3 , device 300 may include a bus 310, a processor 320, amemory 330, a storage component 340, an input component 350, an outputcomponent 360, and a communication interface 370.

Bus 310 includes a component that permits communication among thecomponents of device 300. Processor 320 is implemented in hardware,firmware, or a combination of hardware and software. Processor 320 is acentral processing unit (CPU), a graphics processing unit (GPU), anaccelerated processing unit (APU), a microprocessor, a microcontroller,a digital signal processor (DSP), a field-programmable gate array(FPGA), an application-specific integrated circuit (ASIC), or anothertype of processing component. In some implementations, processor 320includes one or more processors capable of being programmed to perform afunction. Memory 330 includes a random access memory (RAM), a read onlymemory (ROM), and/or another type of dynamic or static storage device(e.g., a flash memory, a magnetic memory, and/or an optical memory) thatstores information and/or instructions for use by processor 320.

Storage component 340 stores information and/or software related to theoperation and use of device 300. For example, storage component 340 mayinclude a hard disk (e.g., a magnetic disk, an optical disk, amagneto-optic disk, and/or a solid state disk), a compact disc (CD), adigital versatile disc (DVD), a floppy disk, a cartridge, a magnetictape, and/or another type of non-transitory computer-readable medium,along with a corresponding drive.

Input component 350 includes a component that permits device 300 toreceive information, such as via user input (e.g., a touch screendisplay, a keyboard, a keypad, a mouse, a button, a switch, and/or amicrophone). Additionally, or alternatively, input component 350 mayinclude a sensor for sensing information (e.g., a global positioningsystem (GPS) component, an accelerometer, a gyroscope, an actuator,and/or image sensor(s) (e.g., camera(s))). Output component 360 includesa component that provides output information from device 300 (e.g., adisplay, a speaker, and/or one or more LEDs).

Communication interface 370 includes a transceiver-like component (e.g.,a transceiver and/or a separate receiver and transmitter) that enablesdevice 300 to communicate with other devices, such as via a wiredconnection, a wireless connection, or a combination of wired andwireless connections. Communication interface 370 may permit device 300to receive information from another device and/or provide information toanother device. For example, communication interface 370 may include anEthernet interface, an optical interface, a coaxial interface, aninfrared interface, a radio frequency (RF) interface, a universal serialbus (USB) interface, a wireless local area network interface, a cellularnetwork interface, and/or the like.

Device 300 may perform one or more processes described herein. Device300 may perform these processes based on processor 320 executingsoftware instructions stored by a non-transitory computer-readablemedium, such as memory 330 and/or storage component 340. Acomputer-readable medium is defined herein as a non-transitory memorydevice. A memory device includes memory space within a single physicalstorage device or memory space spread across multiple physical storagedevices.

Software instructions may be read into memory 330 and/or storagecomponent 340 from another computer-readable medium or from anotherdevice via communication interface 370. When executed, softwareinstructions stored in memory 330 and/or storage component 340 may causeprocessor 320 to perform one or more processes described herein.Additionally, or alternatively, hardwired circuitry may be used in placeof or in combination with software instructions to perform one or moreprocesses described herein. Thus, implementations described herein arenot limited to any specific combination of hardware circuitry andsoftware.

The number and arrangement of components shown in FIG. 3 are provided asan example. In practice, device 300 may include additional components,fewer components, different components, or differently arrangedcomponents than those shown in FIG. 3 . Additionally, or alternatively,a set of components (e.g., one or more components) of device 300 mayperform one or more functions described as being performed by anotherset of components of device 300.

FIG. 4 is a flow chart of an example process 400 for characterizingcells or subcellular components of a specimen for statistical analysis.In some implementations, one or more process blocks of FIG. 4 may beperformed by a characterization platform (e.g., characterizationplatform 210). In some implementations, one or more process blocks ofFIG. 4 may be performed by another device or a group of devices separatefrom or including the characterization platform, such as microscopedevice(s) 230 and/or user device 240.

As shown in FIG. 4 , process 400 may include obtaining a plurality offield images of a specimen, the plurality of field images being capturedby a microscope (block 410). For example, the characterization platform(e.g., using computing resource 222, processor 320, memory 330, storagecomponent 340, input component 350, communication interface 370, and/orthe like) may obtain a plurality of field images of a specimen, asdescribed above in connection with FIGS. 1A-1G. In some implementations,the plurality of field images may be captured by a microscope (e.g., amicroscope device 230).

As further shown in FIG. 4 , process 400 may include processing theplurality of field images to derive a plurality of processed fieldimages, the processing including applying, to the plurality of fieldimages, spatial distortion corrections and illumination-basedcorrections to address deficiencies in one or more field images of theplurality of field images (block 420). For example, the characterizationplatform (e.g., using computing resource 222, processor 320, memory 330,storage component 340, and/or the like) may process the plurality offield images to derive a plurality of processed field images, asdescribed above in connection with FIGS. 1A-1G. In some implementations,the processing may include applying, to the plurality of field images,spatial distortion corrections and illumination-based corrections toaddress deficiencies in one or more field images of the plurality offield images.

As further shown in FIG. 4 , process 400 may include identifying, ineach processed field image of the plurality of processed field images, aprimary area that includes data useful for cell characterization orcharacterization of subcellular features (block 430). For example, thecharacterization platform (e.g., using computing resource 222, processor320, memory 330, storage component 340, and/or the like) may identify,in each processed field image of the plurality of processed fieldimages, a primary area that includes data useful for cellcharacterization or characterization of subcellular features, asdescribed above in connection with FIGS. 1A-1G.

As further shown in FIG. 4 , process 400 may include identifying areasof overlap in the plurality of processed field images (block 440). Forexample, the characterization platform (e.g., using computing resource222, processor 320, memory 330, storage component 340, and/or the like)may identify areas of overlap in the plurality of processed fieldimages, as described above in connection with FIGS. 1A-1G.

As further shown in FIG. 4 , process 400 may include derivinginformation regarding a spatial mapping of one or more cells of thespecimen, deriving the information being based on performing imagesegmentation based on the data included in the primary area of eachprocessed field image of the plurality of processed field images, andobtaining flux measurements based on other data included in the areas ofoverlap (block 450). For example, the characterization platform (e.g.,using computing resource 222, processor 320, memory 330, storagecomponent 340, and/or the like) may derive information regarding aspatial mapping of one or more cells of the specimen, as described abovein connection with FIGS. 1A-1G. In some implementations, deriving theinformation may be based on performing image segmentation based on thedata included in the primary area of each processed field image of theplurality of processed field images, and obtaining flux measurementsbased on other data included in the areas of overlap.

As further shown in FIG. 4 , process 400 may include causing, based onthe information, an action to be performed relating to identifyingfeatures related to normal tissue, diagnosis or prognosis of disease, orfactors used to select therapy (block 460). For example, thecharacterization platform (e.g., using computing resource 222, processor320, memory 330, storage component 340, output component 360,communication interface 370, and/or the like) may cause, based on theinformation, an action to be performed relating to identifying featuresrelated to normal tissue, diagnosis or prognosis of disease, or factorsused to select therapy, as described above in connection with FIGS.1A-1G.

Process 400 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, causing the action to be performed may includecausing the information to be stored in a data structure to enablestatistical analysis of the spatial mapping. In some implementations,causing the action to be performed may further include presenting, fordisplay, a user interface that enables visualization of the informationin conjunction with the plurality of field images and/or the pluralityof processed field images. In some implementations, the user interfacemay include one or more user-selectable options for performing thestatistical analysis.

In some implementations, process 400 may further include deriving amosaic based on the plurality of processed field images, analyzing themosaic to obtain additional information associated with the spatialmapping, and causing the additional information to be stored in the datastructure for facilitating the statistical analysis.

In some implementations, applying the spatial distortion corrections maybe based on a uniform correction model for unwarping images. In someimplementations, applying the spatial distortion corrections may includecross-correlating the areas of overlap.

Although FIG. 4 shows example blocks of process 400, in someimplementations, process 400 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 4 . Additionally, or alternatively, two or more of theblocks of process 400 may be performed in parallel.

FIG. 5 is a flow chart of an example process 500 for characterizingcells or subcellular components of a specimen for statistical analysis.In some implementations, one or more process blocks of FIG. 5 may beperformed by a characterization platform (e.g., characterizationplatform 210). In some implementations, one or more process blocks ofFIG. 5 may be performed by another device or a group of devices separatefrom or including the characterization platform, such as microscopedevice(s) 230 and/or user device 240. In some implementations, a device(e.g., the characterization platform) may include one or more memoriesand one or more processors, communicatively coupled to the one or morememories, configured to perform process 500.

As shown in FIG. 5 , process 500 may include obtaining a plurality offield images of a tissue sample, the plurality of field images beingcaptured by a microscope (block 510). For example, the characterizationplatform (e.g., using computing resource 222, processor 320, memory 330,storage component 340, input component 350, communication interface 370,and/or the like) may obtain a plurality of field images of a tissuesample, as described above in connection with FIGS. 1A-1G. In someimplementations, the plurality of field images may be captured by amicroscope (e.g., a microscope device 230).

As further shown in FIG. 5 , process 500 may include applying, to theplurality of field images, spatial distortion corrections andillumination-based corrections to derive a plurality of processed fieldimages (block 520). For example, the characterization platform (e.g.,using computing resource 222, processor 320, memory 330, storagecomponent 340, and/or the like) may apply, to the plurality of fieldimages, spatial distortion corrections and illumination-basedcorrections to derive a plurality of processed field images, asdescribed above in connection with FIGS. 1A-1G.

As further shown in FIG. 5 , process 500 may include identifying, ineach processed field image of the plurality of processed field images, aprimary area that includes data useful for cell characterization (block530). For example, the characterization platform (e.g., using computingresource 222, processor 320, memory 330, storage component 340, and/orthe like) may identify, in each processed field image of the pluralityof processed field images, a primary area that includes data useful forcell characterization, as described above in connection with FIGS.1A-1G.

As further shown in FIG. 5 , process 500 may include identifying, in theplurality of processed field images, areas that overlap with one another(block 540). For example, the characterization platform (e.g., usingcomputing resource 222, processor 320, memory 330, storage component340, and/or the like) may identify, in the plurality of processed fieldimages, areas that overlap with one another, as described above inconnection with FIGS. 1A-1G.

As further shown in FIG. 5 , process 500 may include derivinginformation regarding a spatial mapping of one or more cells of thetissue sample, wherein the one or more processors, when deriving theinformation, are configured to perform segmentation, on a subcellularlevel, a cellular level, or a tissue level, based on the data includedin the primary area of each processed field image of the plurality ofprocessed field images, and obtain flux measurements based on other dataincluded in the areas that overlap with one another (block 550). Forexample, the characterization platform (e.g., using computing resource222, processor 320, memory 330, storage component 340, and/or the like)may derive information regarding a spatial mapping of one or more cellsof the tissue sample, as described above in connection with FIGS. 1A-1G.In some implementations, the one or more processors, when deriving theinformation, may be configured to perform segmentation, on a subcellularlevel, a cellular level, or a tissue level, based on the data includedin the primary area of each processed field image of the plurality ofprocessed field images, and obtain flux measurements based on other dataincluded in the areas that overlap with one another.

As further shown in FIG. 5 , process 500 may include causing theinformation to be loaded in a data structure to enable statisticalanalysis of the spatial mapping for identifying predictive factors forimmunotherapy (block 560). For example, the characterization platform(e.g., using computing resource 222, processor 320, memory 330, storagecomponent 340, output component 360, communication interface 370, and/orthe like) may cause the information to be loaded in a data structure toenable statistical analysis of the spatial mapping for identifyingpredictive factors for immunotherapy, as described above in connectionwith FIGS. 1A-1G.

Process 500 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the one or more processors, when causing theinformation to be loaded in the data structure, may cause the one ormore processors to cause the information to be loaded in the datastructure to enable statistical analysis, via one or more spatialoperation functions integrated with the data structure, of the spatialmapping.

In some implementations, the one or more spatial operation functions mayrelate to at least one of spatial searches, operations on spatialpolygons, or aggregations subject to one or more of cell phenotypes,subcellular phenotypes, expression levels, or spatial relations. In someimplementations, the one or more spatial operation functions may becapable of representing spatial polygons relating to at least one of aboundary of the tissue sample, architectural features within the tissuesample, morphological features of the one or more cells, or shapes ofmorphological components of the one or more cells. In someimplementations, the one or more spatial operation functions may relateto calculating at least one of intercellular distance, spatial distancesbetween the one or more cells and a boundary of the tissue sample,spatial distances between subcellular components, or spatial distancesbetween one or more of the subcellular components and the boundary ofthe tissue sample.

In some implementations, the information may identify pixels, in theplurality of processed field images, corresponding to outlines of theone or more cells, or center points or subcellular components of the oneor more cells. In some implementations, the information may include dataregarding classification types of the one or more cells. In someimplementations, the one or more processors, when identifying theprimary area in a processed field image of the plurality of processedfield images, may identify a set of pixels, in the processed fieldimage, that is proximate to a center point of the processed field image.

Although FIG. 5 shows example blocks of process 500, in someimplementations, process 500 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 5 . Additionally, or alternatively, two or more of theblocks of process 500 may be performed in parallel.

FIG. 6 is a flow chart of an example process 600 for characterizingcells or subcellular components of a specimen for statistical analysis.In some implementations, one or more process blocks of FIG. 6 may beperformed by a characterization platform (e.g., characterizationplatform 210). In some implementations, one or more process blocks ofFIG. 6 may be performed by another device or a group of devices separatefrom or including the characterization platform, such as microscopedevice(s) 230 and/or user device 240. In some implementations, anon-transitory computer-readable medium may store instructions. In someimplementations, the instructions may include one or more instructionsthat, when executed by one or more processors (e.g., of thecharacterization platform), cause the one or more processors to performprocess 600.

As shown in FIG. 6 , process 600 may include obtaining a plurality offield images of a tissue sample (block 610). For example, thecharacterization platform (e.g., using computing resource 222, processor320, memory 330, storage component 340, input component 350,communication interface 370, and/or the like) may obtain a plurality offield images of a tissue sample, as described above in connection withFIGS. 1A-1G.

As further shown in FIG. 6 , process 600 may include applying, to theplurality of field images, spatial distortion corrections and/orillumination-based corrections to derive a plurality of processed fieldimages (block 620). For example, the characterization platform (e.g.,using computing resource 222, processor 320, memory 330, storagecomponent 340, and/or the like) may apply, to the plurality of fieldimages, spatial distortion corrections and/or illumination-basedcorrections to derive a plurality of processed field images, asdescribed above in connection with FIGS. 1A-1G.

As further shown in FIG. 6 , process 600 may include identifying, ineach processed field image of the plurality of processed field images, aprimary area that includes data useful for cell characterization (block630). For example, the characterization platform (e.g., using computingresource 222, processor 320, memory 330, storage component 340, and/orthe like) may identify, in each processed field image of the pluralityof processed field images, a primary area that includes data useful forcell characterization, as described above in connection with FIGS.1A-1G.

As further shown in FIG. 6 , process 600 may include identifying, in theplurality of processed field images, areas that overlap with one another(block 640). For example, the characterization platform (e.g., usingcomputing resource 222, processor 320, memory 330, storage component340, and/or the like) may identify, in the plurality of processed fieldimages, areas that overlap with one another, as described above inconnection with FIGS. 1A-1G.

As further shown in FIG. 6 , process 600 may include deriving spatialresolution information concerning one or more cells or subcellularcomponents of the tissue sample, wherein the one or more instructions,that cause the one or more processors to derive the spatial resolutioninformation, cause the one or more processors to perform imagesegmentation based on the data included in the primary area of eachprocessed field image of the plurality of processed field images, andobtain flux measurements based on other data included in the areas thatoverlap with one another (block 650). For example, the characterizationplatform (e.g., using computing resource 222, processor 320, memory 330,storage component 340, and/or the like) may derive spatial resolutioninformation concerning one or more cells or subcellular components ofthe tissue sample, as described above in connection with FIGS. 1A-1G. Insome implementations, the one or more instructions, that cause the oneor more processors to derive the spatial resolution information, causethe one or more processors to perform image segmentation based on thedata included in the primary area of each processed field image of theplurality of processed field images, and obtain flux measurements basedon other data included in the areas that overlap with one another.

As further shown in FIG. 6 , process 600 may include causing a datastructure to be populated with the spatial resolution information toenable statistical analyses useful for identifying predictive factors,prognostic factors, or diagnostic factors for one or more diseases orassociated therapies (block 660). For example, the characterizationplatform (e.g., using computing resource 222, processor 320, memory 330,storage component 340, output component 360, communication interface370, and/or the like) may cause a data structure to be populated withthe spatial resolution information to enable statistical analyses usefulfor identifying predictive factors, prognostic factors, or diagnosticfactors for one or more diseases or associated therapies, as describedabove in connection with FIGS. 1A-1G.

Process 600 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the plurality of field images may includemultiple sets of field images captured in different optical bands. Insome implementations, the multiple sets of field images may be capturedin at least thirty-five optical bands.

In some implementations, the one or more instructions, when executed bythe one or more processors, may further cause the one or more processorsto determine optimal thresholds of positivity for the tissue samplebased on the flux measurements, and use the optimal thresholds ofpositivity to derive the spatial resolution information.

In some implementations, the one or more cells may include tumor cellsand immune cells. In some implementations, the plurality of field imagesmay be captured using a multispectral microscope (e.g., a microscopedevice 230).

Although FIG. 6 shows example blocks of process 600, in someimplementations, process 600 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 6 . Additionally, or alternatively, two or more of theblocks of process 600 may be performed in parallel.

Providing an automated pipeline that is flexible and scalable, asdescribed herein, permits the collection and processing of a largeramount of data (e.g., a greater quantity of field images, obtained ineach of multiple optical bands) than possible with prior techniques,which increases analytical system throughput for potential clinical use.In addition, automatically determining optimal thresholds of positivityfor each individual specimen and automated field selection, as describedherein, also increases such throughput. Utilizing machine learningtechniques also streamlines the identification of cell components, suchas cellular nuclei, membranes, cytoplasms, and/or the like. Automatingthe loading of a data structure (e.g., a parallel data structure) withimage segmentation outputs and flux measurements, and providing spatialoperation functions for statistically analyzing the loaded data, reducesor eliminates a need to rely on unwieldy spreadsheets, increases theaccuracy and reproducibility of a fully-automated cell classificationsystem, and facilitates characterizations of interactions at thesingle-cell level (e.g., including spatially-resolved measures ofprotein expression). This provides faster and improved insight intonormal tissue function, disease pathogenesis, inflammatory disease andtumor development, and candidate therapeutic targets.

The foregoing disclosure provides illustration and description, but isnot intended to be exhaustive or to limit the implementations to theprecise form disclosed. Modifications and variations are possible inlight of the above disclosure or may be acquired from practice of theimplementations.

As used herein, the term component is intended to be broadly construedas hardware, firmware, or a combination of hardware and software.

Some implementations are described herein in connection with thresholds.As used herein, satisfying a threshold may refer to a value beinggreater than the threshold, more than the threshold, higher than thethreshold, greater than or equal to the threshold, less than thethreshold, fewer than the threshold, lower than the threshold, less thanor equal to the threshold, equal to the threshold, and/or the like.

It will be apparent that systems and/or methods, described herein, maybe implemented in different forms of hardware, firmware, or acombination of hardware and software. The actual specialized controlhardware or software code used to implement these systems and/or methodsis not limiting of the implementations. Thus, the operation and behaviorof the systems and/or methods were described herein without reference tospecific software code—it being understood that software and hardwaremay be designed to implement the systems and/or methods based on thedescription herein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of possible implementations. In fact,many of these features may be combined in ways not specifically recitedin the claims and/or disclosed in the specification. Although eachdependent claim listed below may directly depend on only one claim, thedisclosure of possible implementations includes each dependent claim incombination with every other claim in the claim set.

No element, act, or instruction used herein should be construed ascritical or essential unless explicitly described as such. Also, as usedherein, the articles “a” and “an” are intended to include one or moreitems, and may be used interchangeably with “one or more.” Furthermore,as used herein, the term “set” is intended to include one or more items(e.g., related items, unrelated items, a combination of related andunrelated items, etc.), and may be used interchangeably with “one ormore.” Where only one item is intended, the term “one” or similarlanguage is used. Also, as used herein, the terms “has,” “have,”“having,” and/or the like are intended to be open-ended terms. Further,the phrase “based on” is intended to mean “based, at least in part, on”unless explicitly stated otherwise.

What is claimed is:
 1. A method, comprising: obtaining, by a device, aplurality of field images of a specimen, the plurality of field imagesbeing captured by a microscope; processing, by the device, the pluralityof field images to derive a plurality of processed field images, theprocessing including applying, to the plurality of field images, spatialdistortion corrections and illumination-based corrections to addressdeficiencies in one or more field images of the plurality of fieldimages; identifying, by the device and in each processed field image ofthe plurality of processed field images, a primary area that includesdata useful for cell characterization or characterization of subcellularfeatures; identifying, by the device, areas of overlap in the pluralityof processed field images; deriving, by the device, informationregarding a spatial mapping of one or more cells of the specimen,deriving the information being based on: performing, by the device,image segmentation based on the data included in the primary area ofeach processed field image of the plurality of processed field images,and obtaining, by the device, flux measurements based on other dataincluded in the areas of overlap; and causing, by the device and basedon the information, an action to be performed relating to identifyingfeatures related to normal tissue, diagnosis or prognosis of disease, orfactors used to select therapy.
 2. The method of claim 1, whereincausing the action to be performed includes: causing the information tobe stored in a data structure to enable statistical analysis of thespatial mapping.
 3. The method of claim 2, wherein causing the action tobe performed further includes: presenting, for display, a user interfacethat enables visualization of the information in conjunction with theplurality of field images and/or the plurality of processed fieldimages, the user interface including one or more user-selectable optionsfor performing the statistical analysis.
 4. The method of claim 2,further comprising: deriving a mosaic based on the plurality ofprocessed field images; analyzing the mosaic to identify a set of fieldsfor which a first feature or a first combination of features satisfies acondition; analyzing the set of fields using a second feature or asecond combination of features; predicting a level of responsiveness toimmunotherapy based on analyzing the set of fields using the secondfeature or the second combination of features; and causing additionalinformation to be stored in the data structure for facilitating thestatistical analysis.
 5. The method of claim 1, wherein applying thespatial distortion corrections is based on a uniform correction modelfor unwarping images.
 6. The method of claim 1, wherein applying thespatial distortion corrections includes cross-correlating the areas ofoverlap.
 7. A device, comprising: one or more memories; and one or moreprocessors, communicatively coupled to the one or more memories,configured to: obtain a plurality of field images of a tissue sample,the plurality of field images being captured by a microscope; apply, tothe plurality of field images, spatial distortion corrections andillumination-based corrections to derive a plurality of processed fieldimages; identify, in each processed field image of the plurality ofprocessed field images, a primary area that includes data useful forcell characterization; identify, in the plurality of processed fieldimages, areas that overlap with one another; derive informationregarding a spatial mapping of one or more cells of the tissue sample,wherein the one or more processors, when deriving the information, areconfigured to: perform segmentation, on a subcellular level, a cellularlevel, or a tissue level, based on the data included in the primary areaof each processed field image of the plurality of processed fieldimages, and obtain flux measurements based on other data included in theareas that overlap with one another; and cause the information to beloaded in a data structure to enable statistical analysis of the spatialmapping for identifying predictive factors for immunotherapy.
 8. Thedevice of claim 7, wherein the one or more processors, when causing theinformation to be loaded in the data structure, are further configuredto: derive a mosaic based on the plurality of processed field images;analyze the mosaic to identify a set of fields for which a first featureor a first combination of features satisfies a condition; analyze theset of fields using a second feature or a second combination offeatures; and predict a level of responsiveness to immunotherapy basedon analyzing the set of fields using the second feature or the secondcombination of features.
 9. The device of claim 7, wherein the one ormore processors, when causing the information to be loaded in the datastructure, cause the one or more processors to: cause the information tobe loaded in the data structure to enable statistical analysis, via oneor more spatial operation functions integrated with the data structure,of the spatial mapping, wherein the one or more spatial operationfunctions relate to at least one of: spatial searches, operations onspatial polygons, or aggregations subject to one or more of: cellphenotypes, subcellular phenotypes, expression levels, or spatialrelations.
 10. The device of claim 9, wherein the one or more spatialoperation functions are capable of representing spatial polygonsrelating to at least one of: a boundary of the tissue sample,architectural features within the tissue sample, morphological featuresof the one or more cells, or shapes of morphological components of theone or more cells.
 11. The device of claim 9, wherein the one or morespatial operation functions relate to calculating at least one of:intercellular distance, spatial distances between the one or more cellsand a boundary of the tissue sample, spatial distances betweensubcellular components, or spatial distances between one or more of thesubcellular components and the boundary of the tissue sample.
 12. Thedevice of claim 7, wherein the information identifies pixels, in theplurality of processed field images, corresponding to: outlines of theone or more cells, or center points or subcellular components of the oneor more cells.
 13. The device of claim 7, wherein the informationincludes data regarding classification types of the one or more cells.14. The device of claim 7, wherein the one or more processors, whenidentifying the primary area in a processed field image of the pluralityof processed field images, are configured to: identify a set of pixels,in the processed field image, that is proximate to a center point of theprocessed field image.
 15. A non-transitory computer-readable mediumstoring instructions, the instructions comprising: one or moreinstructions that, when executed by one or more processors, cause theone or more processors to: obtain a plurality of field images of atissue sample; apply, to the plurality of field images, spatialdistortion corrections and/or illumination-based corrections to derive aplurality of processed field images; identify, in each processed fieldimage of the plurality of processed field images, a primary area thatincludes data useful for cell characterization; identify, in theplurality of processed field images, areas that overlap with oneanother; derive spatial resolution information concerning one or morecells or subcellular components of the tissue sample, wherein the one ormore instructions, that cause the one or more processors to derive thespatial resolution information, cause the one or more processors to:perform image segmentation based on the data included in the primaryarea of each processed field image of the plurality of processed fieldimages, and obtain flux measurements based on other data included in theareas that overlap with one another; and cause a data structure to bepopulated with the spatial resolution information to enable statisticalanalyses useful for identifying predictive factors, prognostic factors,or diagnostic factors for one or more diseases or associated therapies.16. The non-transitory computer-readable medium of claim 15, wherein theplurality of field images includes multiple sets of field imagescaptured in different optical bands.
 17. The non-transitorycomputer-readable medium of claim 16, wherein the multiple sets of fieldimages are captured in at least thirty-five optical bands.
 18. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, when executed by the one or more processors, furthercause the one or more processors to: determine optimal thresholds ofpositivity for the tissue sample based on the flux measurements; and usethe optimal thresholds of positivity to derive the spatial resolutioninformation.
 19. The non-transitory computer-readable medium of claim15, wherein the one or more cells include tumor cells and immune cells.20. The non-transitory computer-readable medium of claim 15, wherein theplurality of field images is captured using a multispectral microscope.